Most modem aircraft utilize digital techniques and software to command control surface positions. The pilot movements of the control instruments, such as the foot pedal or yoke, are translated into electrical signals which are then transmitted to actuators which move the control surfaces. These black boxes which convert movements into electrical signals have software which is critical to the safety of the aircraft.
In order to be sure that these computer systems operate properly on an aircraft, the Federal Aviation Administration (FAA) requires that all software used in a critical system follow a stringent development and testing process. For FAA certification, software is developed and tested according to one of five DO-178 levels: A, B, C, D, or E. Levels D and E require very little analysis and do not require much in order to show compliance. Level C certification is slightly more difficult. At this level of development, software must be traceable to the requirements and test procedures, and the test results must be correlated to the system requirements. This level is mostly concerned with the process of software development.
Level B certification requires that a demonstration of software and implementation be provided such that it can be shown that there is no dead code (unreachable or unexecutable during the test process) in the system. The dead code check is performed from a source code perspective and the test results need to show that each source line of executable code was reached during the test.
Level A certification builds upon level B and places additional requirements on the path coverage testing of the software to be performed in an embedded computer system. Where level B testing only needs to show that every source line was executed and every logic path taken, level A testing must certify that every machine instruction in the object code is executed. In addition, it must be shown that every logic path decision through the software has been exercised exhaustively during the test.
The difference between level B and level A testing is most apparent in the following example using three lines of computer code:
001--IF ((A. and. B). OR (C. and. D)) PA1 002--THEN set E to X PA1 003--set E to Y
For level B testing, it is necessary to show two results: 1) that lines 001, 002, and 003 are executed, and 2) that lines 001 and 003 were executed while 002 was not. For level A testing it is necessary to show that each of the possible combinations that would make the conditional of line 001 true or false. Level B testing may be performed on the source code level. Level A must be performed on the machine code level. Since each source line of executable code is translated into one or more machine instructions by the compiler and linker, a run-time tool that could verify that each machine instruction conditional branch did branch at least once, and at least once did not branch would go a long way towards automating the level A certification testing.
Currently, level A testing is performed on aircraft-critical computer software by manually comparing the results of a test run of the computer program which documents each line of code that was executed, with a link map which is generated at the time the source code is compiled. The person analyzing the code must first determine that every line of code was executed, and that each branch was both taken and then not taken. Because of the amount of time it takes an individual or group of individuals to analyze computer code, especially when aircraft systems are becoming more voluminous and sophisticated, it would be a great advantage to automate this process.
The object of the present invention is to provide a system which automates level A testing of critical software.